United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



10/759.501 



FILING DATE 



01/16/2004 



FIRST NAMED INVENTOR 



Tomomi Takata 



34904 7590 05/12/2009 

CANON U.S.A. INC. INTELLECTUAL PROPERTY DIVISION 
15975 ALTON PARKWAY 
IRVINE, CA 92618-3731 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



CFA00028US 



CHIN, RICKY 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/759,501 


Applicant(s) 

TAKATA ET AL. 


Examiner 
RICKY CHIN 


Art Unit 

2423 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 25 February 2009 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1.6.11.12 and 17-24 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

Claim(s) is/are allowed. 

6) 13 Claim(s) 1.6.11.12 and 17-24 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) D Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20090428 



Application/Control Number: 1 0/759,50 1 Page 2 

Art Unit: 2423 

DETAILED ACTION 

Response to Arguments 

1 . The Applicant's arguments filed February 25, 2009 have been fully considered but 
are not persuasive. 

With regards to claims 1 , 6, 11-12, and 17-18, the applicant argues that Ahmad 
does not teach or suggest obtaining correlation of previous and subsequent scenes. 
The examiner respectfully disagrees. Ahmad teaches of integrating an audio clip based 
on evaluation of the visual recording data (See [0043]) in order to reflect a mood of the 
visual recording (See [0042]). Hence, since the metadata of the visual images are to be 
analyzed to determine an event such as a funeral (See [0035] and [0042]), the audio 
clip is selected based on the correlations of the visual images which would detail the 
type of event such as a funeral or wedding so that the appropriate mood is portrayed. 
Moreover, the examiner relies on Hua for the teaching of obtaining correlation of 
previous and subsequent scenes between which a transition clip is to be inserted and 
obtaining an impression meant to be given to an audience by the correlation to 
determine a transition effect as in claim 1 (See analysis of claim 1 , in particular, Hua, 
col. 16 lines 17-25 which discloses correlating the two sub-shots and integrating a 
transition based on similarities of the two sub-shots). Therefore, it would also have been 
obvious to one of ordinary skill in the art to have modified the teachings of Hua of 
obtaining correlation of previous and subsequent scenes between which a transition clip 
is to be inserted by which the correlation determines the transition effect to be inserted 
and to incorporated obtaining an impression meant to be given to an audience as taught 
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by Ahmad for the mere benefit of better characterizing a transition such that a mood or 
theme is reflected so that a more fitting presentation is displayed to the user according 
to the theme/mood desired. 

Applicant further argues that the transition effect is not determined based on the 
correlation included in the visual recording summary. The examiner respectfully 
disagrees. Ahmad ([0042]-[0045] and [0051]) discloses using transitions that are 
appropriate for particular beat frequencies that produce particular effects to adjust mood 
and feel of the visual summary. Thus, since the audio clips are selected by the 
evaluation of the visual images, and the transition effect is selected to match the beats 
of the audio clip, it is also then that the transition effect is selected based on the visual 
image evaluations as discussed above. Moreover, the examiner relies on Hua (See 
Hua, col. 16 lines 17-25 which discloses correlating the two sub-shots and integrating a 
transition based on similarities of the two sub-shots) for the teaching of the transition 
effect being determined based on the correlation between two scenes and as detailed in 
the above argument. 

Lastly, applicant argues that Ahmad does not teach of wherein the calculating a 
suitability ratio is done by comparing the second impression of and the first impression 
information obtained at the impression obtaining step. The examiner respectfully 
disagrees. Ahmad (See Ahmad, [0042] and [0051]) discloses of evaluating the type of 
transition to use for each pair of clips, and designating particular transitions for different 
effects/moods such as for funerals or parties which is determined from the metadata of 
the content) as well as calculating a suitability score (See [0036] which discloses 
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calculating a suitability score according to any of a combination of evaluations of the 
images). Moreover, the examiner, for further exemplification further relies on Moore for 
the teaching of an additional information storing unit storing in advance the information 
indicating an impression meant to be given to an audience by each transition clip (See 
Moore, col. 8 lines 15-27 which discloses storing categorized sets of transition effects, 
whereby the categorization might include feeling such as conservative, elegant, funny, 
flashy, etc.) Therefore, it would have also been obvious to one of ordinary skill in the art 
to have modified the teachings of the calculating a suitability ratio of Ahmad to 
incorporate the storing in advance the information indicating the impression meant to be 
given to the audience by each transition clip and inserting a transition clip based on 
similarities of the scenes as taught by Hua and Moore for the mere benefit of being able 
to better characterize a fitting transition for the media content to present to the user 
such that it suits the users desires of theme and/or mood. For the reasons mentioned 
above, the rejections are maintained. 

With regards to newly added claims 19-24, the claims have been rejected for the 
reasons set forth below. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1, 6, 11-12, 17-18, and 1 9-24 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Hua et al., US 7,1 27,1 20 in view of Ahmad, US 
2004/0052505, and in further view of Moore, US 7,102,643. 

Regarding claims 1,11 and 12, Hua teaches of an information processing method 
for editing input data (See [Abstract]), comprising: 

an obtaining step of obtaining, from metadata of the data, of two scenes sandwiching a 
position for a transition clip among all scenes in the data and/or object information 
indicating objects existing in the two scenes (See col. 2 lines 52-56 which discloses that 
metadata features are extracted) 

correlation obtaining step of obtaining correlation of the two scenes, object 
information of the two scenes obtained at the obtaining step, from a correlation storage 
unit storing in advance correlation between each object information (See Hua, col. 16 
lines 17-25 which discloses correlating the two sub-shots and integrating a transition 
based on similarities of the two sub-shots; and col. 6 lines 50-col. 7 lines 40;col. 9 lines 
10-20 which discloses the content selection for analyzing the calculated features of the 
metadata extraction module which sets the similarity measure thresholds for 
determining correlations to identify a similarities) 

the first impression and/or effect corresponding to the correlation obtained at the 
correlation obtaining step, from an impression and/or effect storage unit storing, in an 
associated manner, the correlation between the two scenes sandwiching the transition 
clip and the impression and/or effect meant to be given to an audience by the transition 
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clip to be inserted between the two scenes having the correlation (See Hua, col. 16 
lines 15-25 and col. 9 lines 10-23 which discloses that the transition used is determined 
by the similarities of the sub-shots, similarities such as motion intensity. Moreover, the 
sub-shot motion intensity is directly correlated to the music mood (See Hua, col. 11 
lines 25-30 and col. 15 lines 15-30 which discloses that the music mood is matched with 
the motion intensity according to equations and algorithms in the content selection 
module/effect storage unit). Thus, by comparing similarities of the motion intensities of 
the sub-shots, the mood of the sub-shots is also being compared and incorporated into 
the transition being inserted. 

a receiving step of receiving an instruction to specify an arbitrary transition 
clip (See Hua, col. 16 lines 25-30 which discloses randomly choosing a transition clip) 

a transition clip extracting step of extracting at least one transition clip from among 
a plurality of transition clips stored in advance (See Hua, col. 16 lines 30-55 which 
discloses extraction of a transition clip from up to a provided fifteen different types) 

a determining step of determining the transition clip which is specified at 
the receiving step as a transition clip to be inserted into the position being 
sandwiched between the two scenes (See Hua, col. 16 lines 18-50) and 

a processing step of adding a transition effect to the data by using the 
transition clip determined at the determining step (See Hua, col. 16 lines 17-35 which 
discloses the fusion module for integrating via the determined transition). 
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Hua does not explicitly teach of where the metadata of the data includes event 
information indicating a theme of two scenes; a calculating step of calculating a 
suitability ratio indicating suitability of each transition clip; a decreasing order of 
suitability ratio calculated at the calculating step; information associated with a transition 
clip by an additional information storing unit. However, in the same field of endeavor, 
Ahmad teaches of: 

event information indicating a theme of two scenes (See Ahmad, [0035]; [0042]- 
[0043] which discloses evaluating and extracting metadata such as party/funeral and 
objects such as cars/people) 

an impression and/or effect obtaining step of obtaining first impression effect 
information indicating an impression and/or an effect meant to be given to an audience 
by a transition clip to be inserted between two scenes (See Ahmad, [0042]-[0043] and 
[0051] which discloses producing particular emotional effects and mood/feel), 

a calculating step of calculating a suitability ratio indicating suitability of 
each transition clip stored in advance, as a transition clip to be inserted into the 
position being sandwiched between the two scenes (See Ahmad, [0036 which discloses 
calculating a suitability score according to any of a combination of evaluations of the 
images), by comparing second impression and/or effect storing in advance the 
information indicating an impression and/or an effect meant to be given to an audience 
by each transition clip and the first impression and/or effect information obtained at the 
impression and/or effect obtaining step (See Ahmad, [0042] and [0051] which discloses 
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evaluating the type of transition to use for each pair of clips, and designating particular 
transitions for different effects/moods such as for funerals or parties which is determined 
from the metadata of the content); 

a decreasing order of suitability ratio calculated at the calculating step (See 
Ahmad, [0036] and [0047] which discloses displaying clips in increasing or decreasing 
order according to the score); 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have modified the teachings of inserting a transition according to 
similarities of sub-shots as taught by Hua to incorporate where the metadata of the data 
includes event information indicating a theme of two scenes; an impression and/or 
effect obtaining step of obtaining first impression information indicating an impression 
effect meant to be given to an audience by a transition clip; a calculating step of 
calculating a suitability ratio indicating suitability of each transition clip; a decreasing 
order of suitability ratio calculated at the calculating step as taught by Ahmad to be able 
to ascertain visual image characteristics such as events/effects and to be able to 
automatically create video deemed appropriate having a more professional look and 
feel (See, Ahmad, [0019] and [0051]) 

The combination of Hua and Ahmad does not explicitly teach of a displaying step 
of displaying at least one transition clip extracted at the transition clip extracting step, 
receiving an instruction to specify a transition clip from the at least one transition clip 
displayed at the displaying step, and; and of an information associated with a transition 
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clip by an additional information storing unit. However, in the same field of endeavor, 
Moore teaches of a displaying step of displaying at least one transition clip extracted at 
the transition clip extracting step and receiving an instruction to specify a transition clip 
from the at least one transition clip displayed at the displaying step. (See Moore, Fig.9 
(b) and col. 10 lines 35-55 which discloses being able to display and preview transition 
clips for selection) and of an information associated with a transition clip by an 
additional information storing unit (See Moore, col. 8 lines 15-27 which discloses storing 
categorized sets of transition effects, whereby the categorization might include feeling 
such as conservative, elegant, funny, flashy, etc.) 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have modified the teachings of Hua and Ahmad to incorporate 
displaying at least one transition clip extracted at the transition clip extracting step , 
receiving an instruction to specify a transition clip from the at least one transition clip 
displayed at the displaying step, and of an information associated with a transition clip 
by an additional information storing unit as taught by Moore to be able to provide an 
editor the option to preview a transition from among a plurality of transitions before an 
actual transition to ensure the transition is the one desired as well as to be able to better 
characterize a transition clip for insertion which better reflects the users interests. 

Regarding claims 6, 1 7, and 1 8, see analysis of claims 1,11, and 12. 
Furthermore, the claims only differ from claims 1,11, and 12 whereby the transition clip 
extracting step extracts at least one transition clip, based on the suitability ratio 
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calculated at the calculating step, which is unsuitable as a transition clip to be inserted 
(See Ahmad, [0036] and [0047] which discloses a scores for the visual images obtained 
from evaluations indicating desirability and being able to display the clips in an order 
based on the score. Thus, any order may be include decreasing order in which the 
lowest score is displayed. Therefore, it would have been obvious to one of ordinary skill 
in the art to have modified the teachings of Hua to incorporate extracting at least one 
transition clip, based on the suitability ratio calculated at the calculating step, which is 
unsuitable as a transition clip to be inserted as taught by Ahmad to provide further 
flexibility as to better characterize the editor/viewer preference and desires as to the 
type of clip insertions made); an error displaying step of displaying an error message 
(Official Notice is taken by the examiner to note that error message displaying is 
notoriously well-known in the art and would have been obvious to one of ordinary skill in 
the art to modified the teachings of Hua, Ahmad, and Moore to have incorporated an 
error message if the transition clip is deemed inappropriate such as in a transitional clip 
having an upbeat mood for a funeral theme as to alert the editor/viewer.); 
the determining step of determining the transition clip which is specified at the receiving 
step (See Hua, col. 16 lines 15-30 which discloses random selection of a clip as 
specified at the receiving step) from the at least one transition clip displayed at the 
displaying step other than the extracted unsuitable transition clips, as a transition clip to 
be inserted (See Ahmad [0036] and [0047] which discloses a pre-established number of 
candidate images selected by highest score. Thus, the lower scores are deemed 
unsuitable and therefore not displayed after selection. Therefore, it would have been 
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obvious to one of ordinary skill in the art to have modified the teachings of Hua and 
Moore with that of Ahmad to incorporate displaying suitable transitions only as taught by 
Ahmad to accurately select a desired transition which better characterizes an 
editor/viewer desires according to pre-determined factors so that clips deemed as 
inappropriate are not mistakenly selected); 

Regarding claims 19-24, the combination teaches all of the claim limitations of 
claims 1,6,11,12,17, and 1 8. The combination further teaches of wherein the 
additional information storing unit stores in advance the information indicating the 
impression meant to be given to the audience by each transition clip (See Moore, col. 8 
lines 15-27 which discloses storing categorized sets of transition effects, whereby the 
categorization might include feeling such as conservative, elegant, funny, flashy, etc.) 
and an intensity of each impression (See Ahmad, [0042]-[0045] and [0051] which 
discloses using transitions that are appropriate for particular beat frequencies that 
produce particular effects to adjust mood and feel of the visual summary. Thus, by 
varying the tempo and beats a coinciding transition is used, thereby being reflective of 
the intensity of the impression), and wherein the suitability ratio is calculated based on 
the intensity in the calculating step (See Hua, col. 16 lines 17-40; Ahmad, [0047] and 
[0051] which discloses displaying transitions based on beat frequencies to produce 
certain effects to adjust mood and feel and that the clips may be displayed according to 
a score. Hence, since the clips are based on tempo and beat frequencies which portray 
an intensity of a mood/impression, the transition type used is also based on the intensity 



Application/Control Number: 10/759,501 Page 12 

Art Unit: 2423 

of the mood/impression since the transition used is selected to be appropriate for the 
particular beat frequencies that produce the effect and moods). Therefore, it would have 
been obvious to one of ordinary skill in the art to have modified the teachings of the 
storing in advance the information indicating the impression meant to be given to the 
audience by each transition clip and inserting a transition clip based on similarities as 
taught by Hua and Moore and to incorporate having an intensity of each impression and 
calculating a suitability ratio based on the intensity as taught by Ahmad for the mere 
benefit of being able to store the intensity of each impression as to better characterize a 
transition and media content present to the user such that it suits the users theme and 
mood. 

Conclusion 

4. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Contact 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ricky Chin whose telephone number is 571-270-3753. 
The examiner can normally be reached on M-F 8:30-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on 571-272-7296. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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